Skip to content

initial static home directory implementation#5

Draft
patricoferris wants to merge 2 commits intomacosfrom
macos-static-homedir
Draft

initial static home directory implementation#5
patricoferris wants to merge 2 commits intomacosfrom
macos-static-homedir

Conversation

@patricoferris
Copy link
Owner

@patricoferris patricoferris commented Jul 20, 2021 •
edited
Loading

This PR starts to implement the "static home directory" idea where instead of setting the home directory to the build directory we instead have one fixed home directory per build and mount the build directory to this one. This implementation uses ZFS to mount the build directory as I'm not sure there's any way around that.

The main pain-point that this PR does a not-so-great job of fixing is the log file. It is opened in the build directory (i.e. the ZFS dataset directory) and so when that dataset directory is remounted to the fixed home directory... all sorts of errors start to happen. The workaround here is to record the log in /tmp/<id>-log and then copy it over after the build... not ideal... but the code seemed to be working for small examples. The next step will be to try the failing opam-repo-ci ocamlfind jobs.

(note: the "always mounting, never unmounting" ZFS datasets problem is not resolved (yet) in this PR)

@patricoferris
Copy link
Owner Author

patricoferris commented Jul 21, 2021

Successfully built the following spec :)) (I think this was failing before)

((from macos-homebrew-ocaml-4.12)
 (user (uid 1000) (gid 1000))
 (run (shell "ln -f ~/local/bin/opam-2.1 ~/local/bin/opam"))
 (run (shell "opam init --reinit -ni"))
 (env OPAMDOWNLOADJOBS 1)
 (env OPAMERRLOGLEN 0)
 (env OPAMSOLVERTIMEOUT 500)
 (env OPAMPRECISETRACKING 1)
 (run (shell "rm -rf opam-repository/"))
 (copy (src .) (dst opam-repository/))
 (run (shell "opam repository set-url --strict default \"file://$HOME/opam-repository\""))
 (run (shell "opam pin add -k version -yn ppx_deriving_cmdliner.0.6.0 0.6.0"))
 (run (network host)
      (shell "opam update --depexts"))
 (run (cache (opam-archives (target /home/opam/.opam/download-cache)))
      (network host)
      (shell  "opam remove ppx_deriving_cmdliner.0.6.0 && opam install --deps-only ppx_deriving_cmdliner.0.6.0 && opam install -v ppx_deriving_cmdliner.0.6.0;\
             \n        res=$?;\
             \n        test \"$res\" != 31 && exit \"$res\";\
             \n        export OPAMCLI=2.0;\
             \n        build_dir=$(opam var prefix)/.opam-switch/build;\
             \n        failed=$(ls \"$build_dir\");\
             \n        for pkg in $failed; do\
             \n          if opam show -f x-ci-accept-failures: \"$pkg\" | grep -qF \"\\\"macos-homebrew\\\"\"; then\
             \n            echo \"A package failed and has been disabled for CI using the 'x-ci-accept-failures' field.\";\
             \n          fi;\
             \n        done;\
             \n        exit 1"))
 (run (network host)
      (shell "opam update --depexts"))
 (run (cache (opam-archives (target /home/opam/.opam/download-cache)))
      (network host)
      (shell  "opam remove ppx_deriving_cmdliner.0.6.0 && opam install --deps-only --with-test ppx_deriving_cmdliner.0.6.0 && opam install -v --with-test ppx_deriving_cmdliner.0.6.0;\
             \n        res=$?;\
             \n        test \"$res\" != 31 && exit \"$res\";\
             \n        export OPAMCLI=2.0;\
             \n        build_dir=$(opam var prefix)/.opam-switch/build;\
             \n        failed=$(ls \"$build_dir\");\
             \n        for pkg in $failed; do\
             \n          if opam show -f x-ci-accept-failures: \"$pkg\" | grep -qF \"\\\"macos-homebrew\\\"\"; then\
             \n            echo \"A package failed and has been disabled for CI using the 'x-ci-accept-failures' field.\";\
             \n          fi;\
             \n        done;\
             \n        exit 1"))
)

@kit-ty-kate kit-ty-kate mentioned this pull request Jul 21, 2021
Merged
@patricoferris
Copy link
Owner Author

patricoferris commented Jul 30, 2021

cc: @kit-ty-kate I've added the ZFS mounting alternative to #4 here, testing it out on various specs copied from opam-repo-ci. If it all works at some point I'll update the workers and it should mean we have caches and can renable the tests for macos too... in theory...

@kit-ty-kate
Copy link

kit-ty-kate commented Jul 30, 2021

\o/ :tada:

@patricoferris patricoferris mentioned this pull request Sep 12, 2021
Open
@patricoferris patricoferris mentioned this pull request Nov 9, 2021
Closed
6 tasks
@patricoferris patricoferris mentioned this pull request Nov 24, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@patricoferris @kit-ty-kate